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ABSTRACT 

An evaluation committee of Minneapolis Board of 
Education Title I principals initiated the Minneapolis Elementary 
Secondary Education Act (ESEA) Title I Principals • Building Level 
Evaluation Project. The purpose of the project was: (1) to help the 
committee assess the evaluation needs of Minneapolis Title I 
principals; and (2) to study the feasibility of using alternative 
evaluation models for conducting building level evaluation. The five 
phases of the work plan vere: (1) area meetings vith Title I 
principals in Minneapolis North, East and Nest Areas, and from 
parochial schools, to determine interest in building level 
evaluation; (2) in-depth interviews vith ESEA Title I principals vho 
expressed interest in building level evaluation; (3) literature 
search for evaluation models relating to building level evaluation; 
(4) preparation of an evaluation *'mena" consisting of a simple, 
nontechnical pamphlet designed to stimulate discussion on building 
level evaluation; and (5) preparation of a final report of the 
project. Through a series of meetings, questionnaires, and 
interviews, it was possible to ascertain the interests of Title I 
principals in building level evaluation. Their main areas of concern, 
in order of frequency, were: (1) overall program effects on pupils; 
(2| attitudes of staff, pupils, and parents toward the program; and 
(3) best allocation of Title I funds. (Author/BJG) 
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EXECUTIVE SUMMARY 



An evaluation committee of Minneapolis Board of Education Title I 
principals has been concen\ed with possibilities for conducting evalu- 
ation at the Individual building level. Committee members perceive 
building level eval\iatlon as a means for Improving Title I programs, 
facilitating decision making, and encouraging the participation of 
staff, parents, and community members In educational planning and 
declslon-^maklng activities*. 

At a planning meeting In April, 1975, the committee Initiated the 
Minneapolis ESEA Title I Principals* Building Level Evaluation Project. 
The purpose of the project, conducted by Educational Testing Service, 
was: (1) to help the committee assess the evaluation needs of - 
Mlxmeapolls Title I principals; and (2) to study the feasibility of 
using alternative evaluation models for conducting building level eval- 
uation* The project Involved cooperative activities by Title I princi- 
pals, the Department of Itesearch and Evaluation, and Educational Testing 
Service. 

The work plan for the project was developed cooperatively and In- 
cluded five phases: 

(1) Area meetings with Title I principals In Minneapolis 
North, East and West Areas, and from parochial 
schools, to determine Interest In building level 
evaluation; 

(2) In-depth Interviews with ESEA Title I principals \Aio 
expressed interest in building level evaluation; 

(3) Literature search for evaluation models relating to 
building level evaluation; 

(A) Preparation of an evaluation "menu" consisting of a 
•is^le, nontechnical paiq>hlet designed to stimulate 
discussion on building level evaluation; and 

(5) Preparation of a final report of the project. 

Through a series of meetings, questionnaires, and interviews, it 
was possible to ascertain the interests of Title I principals in build- 
ing level evaluation. Their main areas of concern, in order of fre- 
quency, were: (1) overall program effects on pupxls; (2) attitudes of 
staff, pupils, and parents toward the program; and (3) best allocation 
of Title I funds. The kinds of building level desired by Title I prin- 
cipals were, in rank order: (1) product evaluation of Ti^le I pro^ 
graas; (2) formative evaluation of relatively new programs; (3) effi- 
ciency evaluation, such as a cost-effectiveness study; and (4) input 
evaluation, such as assessment of school resources. 



The evaluation "menu". entitled. About Building Level Evaluation^ 
was prepared following an Intensive literature search on evaluation 
nodels. The pamphlet was reviewed by members of the Committee and the 
Department of Research and Evaluation, and was revised In accordance 
with suggestions received. The "menu" provides a general Introduction 
to building level evaluation, presents and briefly explains some gen- 
eral evaluation models which could be applied at the building level, 
and provides an overview of activities and considerations in conduct- 
ing building level evaluations. 

Two building level evaluation studies are recommended for first 
year pilot evaluation projects. They are: (1) a pilot evaluation 
project emphasizing pupils* acquisition of basic skills, to follow a 
summatlve evaluation model; and (2) a pilot evaluation project with 
community participation which would focus on pupils* affective devel- 
opment, to follow a formative evaluation model. The final report 
suggests some possibilities for the design, instrumentation, and sub- 
jects for each of the two recommended pilot studies. The final report 
also suggests some guidelines for establishing an effective relation- 
ship between a Title I pilot evaluation school and an outside 
consultant. 
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BACKGROUND OF THE PROJECT 

ESEA Title I programs In the public schools of Minneapolis are monl** 
tored and evaluated systematically In accordance with federal guidelines. 
Evaluation, In general, has been conducted on a dlst rlct-*vide basis for 
the purpose of assessing overall Impact of Title I projects. Relatively 
little attention has been paid to assessing Title I projects at the Indi- 
vidual building level. Increasingly, principals of elementary schools 
which receive Title I funds have been expressing Interest In conducting 
evaluation studies of compensatory educational projects Initiated within 
their own schools. 

In April, 1975, an evaluation committee of Minneapolis Title I prin- 
cipals* held a planning meeting to consider possibilities for developing 
one or more models for building level evaluation. Educational Testing 
Service, (ETS), a nonprofit educational research and measurement organi** 
Mtlon, was selected to help the committee in assessing evaluation needs 
of Minneapolis Title I principals and studying the feasibility of using 
alternative evaluation models. 

Heabers of the evaluation comnittee felt that the purpose of build- 
ing level evaluation would primarily be to provide timely and relevant 
Information to Title I principals, faculty, and parents so that school*- 



«!fr. George Haafcenson, Principal, Bancroft School 
Mr. Richard J. Hanson, Principal, Calhoun School 
Mr. Jack Ott, Principal, Lowell School 
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Initiated projects could be continuously Improved. They believed that 
such locally Initiated* low profile evaluation studies could facili- 
tate decision making without the threat of undue publicity which some- 
times accompanies the widespread dissemination of cltywlde evaluation 
findings. Furthermore, the committee members perceived building level 
evaluations as being vehicles suitable for encouraging broad based 
^participation in educational planning and decision making on the part 
of staff, parents, and Interested community members. During the 
April 25th meeting, an action plan was drawn up for cooperative work 
on the part of Title I principals, the Department of Research and 
Evaluation, and ETS. The section on scope of work, which follows, 
describes the functions of each of the participating groups. 
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SCOPE OF THE PROJECT 

The work plan which resulted from the Initial planning meeting of the 
evaluation committee. Department of Research and Evaluation^ and ETS staff, 
defined the scope of the Minneapolis ESEA Title I Principals' Building 
Level Evaluation Project. The plan included the following steps:- 

(1) Area Meetings. During the month of May, group interviews would 
be held with Title I principals in the North, East, and West Areas of Min- 
neapolis. During the course of the meetings, the purpose of the project 
would be described by the project director, and each Title I principal 
would be requested to complete a short questionnaire. The questionnaire 
asked each principal to indicate his or her interest in building level 
evaluation, and the kind of evaluation that might be particularly useful 
for Title I programs. The result of the Area meetings would be a prelim- 
inary list of principals with expressed interest in, and need for, building 
level evaluatioL of Title I projects. 

(2) ESEA Title I Principal Interviews. A detailed interview sched- 
ule would be developed, based on the preliminary information gained from 
the Area meetings. Each Title I principal who expressed any degree of 
Interest in the notion of building level evaluations would be interviewed. 
The purpose of the interviews would be to examine, in depth, the types of 
Title I projects initiated at the building level, and the kinds of evalu- 
ation Title I principals are considering. Tht result of the interviews 
would be a document which susmarizes school initiated Title I programs, 
and the perceived need for Title I building level evaluations. 
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(3) Literature Search. A literature search on building level evalu- 
ation models would be conducted » including an ERIC coiq>uter** assisted 
search of recent publications. The purpose of the literature search would 
be to Identify possible evaluation models which could be applied in the 
Minneapolis Title I elementary school setting. The results of the litera- 
ture search, in combination with data from interviews and group meetings, 
would be a building level evaluation "menu." The concept of a 'Haenu'* of 
evaluation models was initi&'red by ^he evaluation committee. It was felt 
that a shorty simply written, nontechnical pamphlet could be used by prin- 
cipals, faculty, parents, and community members as a starting point in ex- 
ploring possibilities for evaluating school initiated projects. 

(4) Draft J Review jond Revision of Evaluation "Menu. " The building 
level evaluation "menu" would be written in draft form and circulated 
among menibers of the evaluation committee and staff' of the Department of 
Research and Evaluation for comments. Revisions would be made in line 
with recommendations of the reviewers. The final evaluation "menu" would 
be printed to provide several copies for each interested Title I princi- 
pal. 

(5) Final Report. A final summary report would be prepared, describ- 
ing the entire project. It would include the revif^ed "menu" and recommen- 
dations for trial implementation of building level evaluation in the Min- 
neapolis Title I elementary schools. 
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BESULTS 

Aaaessment of Building Level Svaluation Needs . 

Of the 32 Title I elementary schools' and 5 Title I parochial schools* 
principals > how many would be willing to commit school resources and work 
cooperatively with the Department of Research and Evaluation or other eval- 
uation specialists to conduct a pilot evaluation program? Through a series 
of meetings, questionnaires, and interviews. Title I principals were asked 
to indicate the extent and kinds of their evaluation needs. 

(1) Title I Principals' Area Meetings 

During the month of May, the project director with Title I 
principals in East, North, and West areas as well as from Title I 
parochial schools. The purpose of the Building Level Evaluation 
project was described briefly, and each Title I principal was 
asked to complete a preliminary questionnaire (Exhibit 1 In 
Appendix) indicating whether there is interest in building level 
evaluation. Response frequencies are shown below. 

Table 1 

FREQUENCY OF TITLE I PRINCIPALS* RESPONSES REGARDING 
INTEREST IN BUILDING LEVEL EVALUATION 

••Not 

Number of "Interested" Interested" "Undecided" **No Response" 

Area Title I Schools Principals Principals Principals Principals 

East 11 8 .3 

North 14 3 2 9 

West 7 2 5 

Parochial ^ _5 

Total 37 16 2 2. 17 
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Vtilch building level projects did the principals want to 
evaluate? And what kind of evaluation did they have In mind? 
A tally of responses to the preliminary questionnaire showed 
a broad range of Interests » with emphasis In the basic skills 
areas: reading or language arts and arithmetic. Table 2 
shows the frequency of responses to each category • It was 
not clear 9 however , whether the principals were concerned 
specifically with bulldlng-lnltlated Title I projects, or 
with the distrlct-wlde projects. 

Table 2 

FBEQUENCY OF TITLE I PRINCIPALS' RESPONSES REGARDING 
INTEREST IN AREA AND TYPE OF EVALUATION 

Type of Evaluation: Input Formative Summatlve Efficiency 



Area to be evaluated: 
Language Arts 
Reading 
Math Lab 
Basic Skills 
Career Awareness 
Teacher Effectiveness 
Aids Effectiveness 
Open School 
Social Studies 
Science. 

Moral Development 
Parent-Teacher Relations 



1 
2 
1 
1 



2 

3 
4 



1 
1 



1 
1 



(2) Principal Interviews 

Soon after the Area meetings, letters were sent to each Title I 
principal, with the exception of two principals who were firm in 
stating they had no Interest in building level evaluation. The 
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letters (Exhibit 2 in Appendix) described the purpose of the 
Title I principals* project, and requested their cooperation 
in exploring their interests and needs for evaluating building- 
initiated Title I projects. Telephone calls were made to each 
of the 35 Title I principals as followup to the letters. A 
total of 20 principals were interested enough in the notion of 
building level evaluations to consent to in-depth interviews* 
A detailed summary report of the interviews was submitted to 
the Department of Research and Evaluation in July. A summary 
of the interim report follows below. 

Purpose of Principal Interviews. The semi-structured inter- 
views with 20 Title I principals were intended to collect 
some answers to the following questions: 

1. What are the specific building level evaluation needs 
in each building: who» what, ^en, where, and how do 
the principals want to evaluate? 

2. What available data are there, which would be relevant 
to their evaluation needs, and what additional data 
will be necessary? 

3. What is the level of understanding of the notion of 
building level evaluation, and how deep and broad are 
the evaluations they envision? 

4. Who, in the principal's opinion, should share in the 
planning phase of any kind of building level evalua- 
tion? 

5. To whom should building level evaluation findings be 
disseminated? 

15 



Inatrumentaiion. A Principal Interview Schedule (Exhibit 3- 
in Appendix) was developed as guide to assessing the Title I 
principals* knowledge of and interest in building level eval- 
uations. The interviews were conducted in a relaxed » inform- 
al atmosphere in their schools. Every effort was directed 
towards pursuing each principal's own evaluation interests 
and concerns. Ample opportunities were afforded each princi- 
pal to express freely his own views and ask questions about 
evaluation at the building level as opposed to district-wide 
evaluation studies. 

Interviei) Findings. Considerable variation was observed 
among the principals in terms of their levels of sophistica- 
tion with regard to measurement and evaluation. There was 
some confusion as to which Title I projects were initiated 
by the school, and which were originated from the central of- 
fice. It was also sometimes difficult to discriminate Title 
I sponsored components from non-Title I components of educa- 
tional programs within schools. Interpretation of the prin- 
cipals' responses must therefore be made in the light of some 
of these observed ambiguities. 

The kinds of evaluation desired by the principals were rank- 
ordered according to frequency of respo^^ses. Product evalua- 
tion of Title I programs was the top choice of these admini- 
strators. Formative evaluation of relatively new programs 
vaa also a popular request. Efficiency evaluation, such as 
the relative cost effectiveness of devoting Title I monies 
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to resource teachers versus teacher aides* was third. Input 
evaluation, the assessment of the resources of the schools » 
was last In terms of principal Interest. 

The relative numbers of Central Office programs and Building 
Level Programs according to principal reports In the 20 
schools visited are shown below. It must be noted that the 
distinction between "central office" and '*bulldlng'*^ Title I 
programs was far from being clearly drawn. Any Title I eval- 
uation conducted at the building level will probably Involve 
assessing the total Impact of Title I funds, regardless of 
the origins of the programs. 

Table 3 



FREQUENCIES OF. CENTRAL OFFICE AND BUILDING TITLE I 
PROGRAMS IN 20 TITLE I ELEMENTARY SCHOOLS 



Central Office Level 

Program Frequency 

Instructional 

Resource Center 13 

Mathematics 9 

Basic Skills 

Center 8 

Reading 8 

Title I Resource 

Teachers 7 

Talking Typewriter 3 

ABC 2 



Building Level 
Program Frequency 

Teacher Aides 15 

Resource Teachers 8 

Math (labs 

included) 14 

Reading 12 

Clerical 3 

System 80 3 

Modified Open 1 
School 
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The chief areas of concern with respect to building level eval- 
iiations varied among the respondents. Top priorities were 
given to oyerall program effects on students, attitudes of 
staff, students and parents toward programs, and the best allo- 
cation of Title I funds. Frequency of the principals' main 
concerns are listed below* 

Table 4 

FREQUENCY OF TITLE I PRINCIPALS* RESPONSES REGARDING 
MAIN CONCERNS FOR BUILDING LEVEL EVALUATIONS 

Principals' Main Concerns Frequency of Response 

1» What is the overall effect of 

Title I Programs on S4:udents? 17 

2. Attitudes toward Title I 

Programs : 15 

a. Teachers /Aides 7 

b. Students 6 

c. Parents 2* 

3. How should building level 
Title I funds be used (i.e., 

resource people, aides, etc.)? 9 

4. Formative evaluation of Title I 6 

5. Building-wide program evaluation 5 

6. Staff development 3 

7. Conmunity involvement 4 

8« Summative evaluation to justify 

building level Title I funds 2 

The interviewed Title I principals were generous with their time, 
and candid and affirmative in expressing their needs and concerns. 
They were quite certain in '"what," "who," and "when" they wanted 
In terms of building level evaluation. There was less certainty 
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In the Issue of "how" such evaluation should proceed. They 
were agreed that an evaluation model that would work well at 
one school could be transported to similar programs at other 
schools. Their awareness of shared Interests and comon pro<* 
grams (at least In name) among some schools would work In 
favor of conducting a few pilot projects. Nor are the kinds 
of data presently available from each of the schools markedly 
different from each other. Many of the principals were look-* 
Ing to the possibility of a building level evaluation project 
as an opportunity for the administration, staff, and parents 
to learn and work together In cooperation with an evaluation 
specialist or coordinator from the central office. The find- 
ings of such a study would not necessarily be limited to the 
building, since a number of principals felt their programs 
would survive the scrutiny of an exacting assessment. 

Given the willingness on the part of a number of principals to 
participate in a pilot building level evaluation project, it 
remains to decide what evaluation models might be feasible for 
the interested schools, how a pilot evaluation project could 
be undertaken, and what schools might be fruitful sites for a 
pilot project. If the level of enthusiasm about building 
level evaluation is a prime criterion for the choice of cooper- 
ating schools, the following two lists of schools and princi- 
pals should be considered in hierarchical order. 

(a) Principals who were enthusiastic about building 
level evaluation, and were primarily concerned 
with assessing basic skills: 

" 19 



School 
Bancroft 

Webster 

Intermediate 

Burroughs 

Corcoran 

Northrop 



Principal 
Mr. Haakenson 

Mr. Gustafson 

Mr* McConvllle 
Mr. Schumack 
Mr. S. Buchanan 



Focus of Evaluation 

Narr.i^ focus » summatlve evaluation 
for accountability purposes 

Formative evaluation as diagnostic 
tool for staff 9 summatlve evalua*- 
tlon of total program 

Formative and summatlve evaluation 

Formative and summatlve <svaluatlon 

Formative evaluation of reading 
program 



(b) Principals who were enthusiastic about building level eval- 
uation, but were concemeu with affective outcomes as well as 
basic skills: 



School 
Clinton 

Agassis 
Holland 



Principal 
Mr. Eckland 

Mr. Modell 
Ms. Hlgley 



Focus of Evaluation 

Formative evaluation of open school, 
reactions of pupils, teachers, 
parents, community 

Staff and student attitude, learning 
environment 

Open school t evaluation of program 



A ''Menu" of Evaluation Models . 

In addition to a survey of Title I principals' evaluation needs, a 
•econd building level evaluation aid which the Title I principals' com- 
■Ittee requested was a brief '"menu" describing various evaluation models 
which might be applicable to programs in their schools. The purpose of 
0uch a "menu" would be to serve as a starting point for discussions about 
building level evaluation among the principal, teachers, and interested 
parents, with the guidance of Department of Research and Evaluation pro- 
fessionals. While there would not be enough information on such a "menu" 
for the school themselves to design for themselves an evaluation study, 
there might be enough ideas to provoke a lively and informed discussion 
aaong Interested parties. 
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Accordingly 9 a "menu" for stimulating appetites for building level 
evaluation was written after an extensive literature search* reviewed 
by committee members and Research and Evaluation staff , and revised In 
accordance with their suggestions. The "menu" was designed to provide 
a general Introduction to building level evaluation, present and brief- 
ly explain some general evaluation models , and provide an overview of 
activities and considerations In conducting building level evaluations. 

The library search and ERIC title search yielded an enormous amount 
of material In evaluation of compensatory education programs In general, 
a substantial number of publications devoted to evaluation models In par- 
ticular, but a relative paucity of references on models for evaluating 
programs specifically at the building level. Of necessity, the little 
pasq)hlet. About Building Level Evaluation 9 dealt with general evaluation 
models which could be applied at the building level. Yet, because an 
elementary school Is a self-contained microcosm, a well designed evalua- 
tion study may well yield mor(% detailed and reliable local Information 
than can be gained from the large scale dlstrlct'-wlde evaluation efforts. 
The following section contains some recommendations for pilot projects 
to try out one or two evaluation models at cooperating schools. 

The following list shows principals with shared areas of concern. 
Any product evaluation project which appears to be successful In one 
of the buildings would be transportable to the others with the same 
priority areas. 
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List 1 



PRIORITY AREAS OF CONCERNS IN BUILDING LEVEL EVALUATION 
(BY AREA IN ALPHABETIC ORDER OF NAME OF PRINCIPAL) 



Priority Areas of Concerns /Schools 

1. Evaluation of Open School Programs 

Harrison 

Wilder 

Standlsh 



2. 



Name of Principal 



Mr. Bentson 
Mr. Eckland 
Mr. Flat turn 



Comparative effectiveness of teacher aides, resource teachers* other 
ways of using Title I funds. 



Northrop 
Irving 
Kenwood 
Standlsh 

Webster Intermediate 

St. Bridget 

Bethune 

Seward 

Jefferson 



Mr. S. Buchanan 
Mr. W. Buchanan 
Mrs. Danahy 
Mr. Flattum 
Mr. Gustafson 
S. Delores Leuer 
Mr. McDonough 
Mr. Monson 
Mr. Pollard 



3. Attitude and Motivation among pupils, parents, teachers 



Cleveland 
Madison 
Bethune 
Agassi z 

North Area Learning Center 

4. Mathematics Achievement 
Greeley 

Webster Intermediate 

Bancroft 

Holland 

St. Bridget 

Madison 

Seward 

5. Reading 

Bancroft 
Longfellow 
Agasslz 
Seward 



Mr. Drolsum 
Mrs. Lincoln 
Mr. McDonough 
Mr. Mbdell 
Mr. Ramherg 



Mr. Gall 

Mr. Gustafson 

Mr. Haakenson 

Mr. Hlgley 

S. Delores Leuer 

Mrs. Lincoln 

Mr. Monson 



Mr. Haakenson 

Mr. McConvllle 

Mr. Modell 

Mr. Manson 
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Section 4 
RECOMMENDATIONS 

Introduction. 

The genuine Interest shown by a majority of the Title I principals, 
the existence of common areas of Interest among several buildings , and 
the availability of cltyvlde and Title I assessment data combine together 
to suggest that relatively modest trial projects might have future appll** 
cations to a number of Title I schools. The levels of knowledge mani- 
fested by the principals, however, indicate that substantial professional 
support would be needed from outside sources if valid and transportable 
building level evaluation model projects are to be developed. 

If one or two pilot evaluations were to be carried out, what pro-* 
jects should be tried first? Who would bear the responsibility of the 
different components of the task? How can the work be divided between 
the building staff and community on the one hand, and the necessary out-* 
side professional consultant on the other? The following suggestions are 
based on th^i findings from the questionnaires and interviews. Realities 
of^ the educational scene in Minneapolis, however will dictate whether 
even modest building level evaluation projects can be carried out* 
Ideally, an evaluator from Research and Evaluation would be assigned to a 
cooperating school, probably on a part-^time basis. Since resources for 
such projects are limited in the schools as well as in the central office, 
the Initial trial project(s) should be modest in scale. A pilot project 
■ust be very carefully designed and carried out so that the model can be 
applied to other schools if restilts warrant it. Help in evaluation design 
and Instrumentation might also be explored from nearby universities or 
other outside sources. 
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Recommendations for Pilot Evaltjatlon Projects , 

Initially, no more than two pilot evaluation studies can be feasibly 
undertaken. The site of pilot evaluation projects should be selected on 
the basis of enthusiasm and commitment on the part of not only the prln- 
dpaly but of the staff and parents or community members as well. By 
keeping the pilot projects modest In scale and low In profile. It should 
be possible to try out the concept of building level evaluation without 
unduly penalizing the pioneer schools who volunteer to undertake building 
level evaluation. 

It should be reasonable to assess pupil progress In two distinct 
dimensions during the first year pilot evaluation projects. One might 
concentrate on the acquisition of basic skills. The other might look 
primarily at affective variables, such as pupil attitudes to school, 
parental support of school goals and practices, community support, 
teacher attitude, or classroom climate. Some possib\llities for design, 
instrumentation and samples for each of the two recommended pilot 
studies are discussed below. 

(1) A pilot project with emphasis on evaluating ihe acquisition of 
basic skills would follow a sunmative evaluation model . Such a project 
would rely heavily on citywide standardized test data. Quality of the 
study would depend on the coiq>leteness and reliability of pupil records. 

DeBign. An experimental or quasi-*experimental design could be 
used in cooiparlson with two or more ways of spending Title I monies at 
the school level. For example, it would be possible to compare achieve-* 
mmat of pupils with classroom teachers and resource teachers in reading 
or math; and that of pupils with teachers and aides. It may also be 

16 

24 



ERIC 



possible to compare achievement of pupils with teachers and aides in com- 
bination, with or without the additional tool of teaching machines. Deci- 
sion on design will be dictated by the possibility for random assignment 
of individuals and classes to treatment, availability of control groups, 
and data files. 

Subjeeta. In order to keep the project within manageable scale, 
limited classes of pupils in one or two grade levels should constitute the 
subjects of the pilot evaluation. If the decision is to conduct a longi- 
tudinal study, students in grades 5, 6, and 7, might be good choices, 
since annual cityid.de test data on the same instruments would be available 
from the past three years for many of the students. 

Data souroes. In addition to citywide test data, the Minneapolis 
Reading and Mathematics Criterion-Referenced Tests, if recorded with care, 
could be quantified for program evaluation purposes. Matched longitudinal 
data on individual pupils might be available from the Educational Service. 
Center. Item analysis of group performance in specific achievement areas 
can yield useful information on the strengths and weaknesses of specific 
educational programs. 

Major aaaietanae requirements. Since completeness and accuracy 
of pupil achievement records are prime requirements for this pilot project, 
major effort should be devoted to making certain that pupil records are as 
conrplete as possible. Those with reading achievement test scores on in- 
struments other than the Gates series might be included by using the 
Anchor Study tables to translate other reading test scores to Gates equiv- 
alent Fzorea. A consistent method for quantifying and recording pupil 
progress with the Minneapolis Criterion-Referenced Tests would add depth 
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to the information available. Consultant assistance such as from Research 
and Evaluation would probably be nc. ^^.ary to set up a student information 
system, which car be continued with the help of clerical aides or volun- 
teers in the cooperating schools. If a system could be developed to de- 
liver longitudinal data on individual pupils, stored in the central files, 
the task of conducting longitudinal studies in a building would be light- 
ened. Data analysis and interpretation could also require outside assist- 
ance. Here, too, systems developed for the pilot buildings should be 
applicable to other Title I schools with similar evaluation needs. 

(2) A Pilot evaluation project with community part icipation focusing 
on the affective development of pupils, based on a form ative evaluation 
model. Such a pilot project would need substantial support from Research 
and Evaluation staff or other consultants in instrument design, staff, and 
community training, as well as design and data analysis. 

Design. A quasi-experimental design using nonrandom comparison 
groups is a reasonable possibility. Several open schools are interested 
in some kind of building level evaluation in the affective domain. There 
are. In nearby locations, "contein)orary" and "continuous progress" sister 
schools which serve similar populations. Since the Southeast Alternative 
Schools have had considerable experience in conducting formative evalua- 
tion of nontradltional settings, their experiences to date with design 
and instrtiaentation may be applicable to a Title I pilot- school. 

Stdbjeata. In addition to pupils, school staff and parents can 
also be considered reasonable subjects for assessment of school climate 
and community relations. Again, the population to be considered should 
be severely limited. A thorough study of one or two grade levels, and 
selected classrooms would be preferable to a large school-wide effort. 
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Data Sources. Appropriate Instruments, such as observation 
systems » questionnaires, or Interview schedules, could be designed or 
adapted from other evaluation projects, A number of ETS bibliographies 
on measures of affective development are available for review from the 
Department of Research and Evaluation, It may be possible to conduct 
time consuming data collection, such as observations and Interviews, 
with the assistance of volunteer parents or conmKunlty members. A con-* 
slderable training effort will be necessary for school evaluation vol- 
unteers, but restilts might be reaped In terms of Improved community 
participation, as well as reliable data about human relations and 
attitudes to school. 

Major asaiatanae requiremBnts. Substantial professional sup- 
port will be needed to Identify the specific affective areas to be eval- 
uated, delimit the size of the study, design or adapt Instruments needed, 
and train nonprofessional volunteers to collect data reliably. If par- 
ents and teachers are to have considerable control In the decision and 
Implementation of such an evaluation, professional consultants may have 
to devote several meetings to explanations and question answering before 
a decision on building level evaluation can be reached. However, the 
resulting community participation model may be transportable to other 
schools. 

Recommendations for Working Relationship Between Title I Pilot Evaluation 
School and Consultant . 

Can an effective working relationship be established between a 

Title I pilot evaluation school and the outside consultant, such as an 

evaluation specialist from the Department of Research and Evaluation 
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or other source? A collaborative project should follow a number of 
simple guidelines: 

(1) Eiophaslze continuous Joint planning, and the establishment of 
direct and open communications between the school and the consultant, 
by designating an evaluation coordinator at the school, who will be the 
prime contact person. 

(2) Devote Initial meetings to the establishment of clearly delin- 
eated division of responsibilities between the school and the consultant 
which will result In each party contributing significantly to the pilot 
proj ect. 

(3) Everyone at the school who will be involved in the pilot pro- 
ject should have a thorough understanding of the purpose and procedures 
involved in the proposed building level evalu?.tion. 

(4) Make provisions for early decisions on evaluation design and 
instrumentation, so that data collection can be spaced during the school 
year, and results can provide effective feedback for program Improvement. 

(5) Analysis and interpretation of findings will be held in confi- 
dence between evaluator and the cooperating school. Decision on dissem- 
ination should be left in the hands of the building principal. 
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Exhibit 1 



(PRELIMINARY PRINCIPAL QUESTIONNAIRE) 
EVALUATION INTSKEST QUESTIONNAIRE 

Principal School Area 



!• Have you undertaken any building level evaluation i now or In the past? 
If your answer is yes, please give a brief description of the project(s) 
and the evaluation design. 



2« Have you programs or projects Initiated at the building level that you 
would like to evaluate? 

If your answer is yes, please list and give brief description of the 
program(s) • 

Title 

Location 

Grades served 

# pupils served ■ 

# years in operation . 



3« What kinds of evaluation would interest you the most? Please rank the 
four types listed below from 1 (most desirable) to 4 or 5 (least 
desirable). 

Input evaluation ; assessing the level of resources and effort 

put into the project. 
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Formative evaluation ; assessing and evaluating the process or 
the way the project is implemented^ for project improvement. 

Product evaluation ; evaluating the end results of a project* 
to see if the goals have been attained. 

Efficiency evaluation ; evaluating the relationship of the re- 
sults of a project to the costs Incurred^ to assess the cost 
effectiveness of a project. 

Other (please specify); 
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Exhibit 2(A) 



SAMPLE LETTER OF REQUEST FOR PRINCIPAL INTERVIEW 



EDUCATIONAL TESTING SERVICE 



BVANSTON OFFICE 



Arm Cede JJ2 
9-7700 



Kay 29, 1975 



Dear Mr. 



Lut veek» I had a chance to meet with you during the Area Title I prin- 

cipals' meeting. At that time, you had expressed Interest In exploring the 
possibilities for building level evaluations. 

As a second step of the Principals* Committee project on developing models for 
building level evaluation, interviews will be conducted with every Interested 
Title I principal. ETS, a non-profit organization devoted to measurement, 
evaluation and research in education, has been contracted by the Minneapolis 
Board of Education to work with the Principals* Committee. During the next 
week cr two, you will be contacted by my colleague. Bob Reineke. We would be 
very grateful if you could consider, in the meantime, your own building evalu- 
ation needs for next year in terms of the following: 

1* The kind of evaluation you prefer. 

2. The program(s) you would like to evaluate. 

3. The data available and the data needed for 
your, building. 

4* . The people in your school or conmunlty who 
should be involved in the planning and data 
collection phases of any evaluation to be 
undertaken. 

5* To whom the findings of any evaluation should 
be. reported. 

Hr. Reineke will explore these issues with you during a brief interview at your 
convenience. We hope to come up with several evaluation models which would 
•4dress the specific evaluation needs of the Title I principals. 

Sincerely yours, 
Jayjia Hsia, Ph.D. 



STREET • BVANSTON. II^LINOIS 60201 
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Exhibit 2(B) 



SAMPLE LETTER OF REQUEST FOR PRINCIPAL INTERVIEW 



EDUCATIONAL TESTING SERVICE 



BVANSTON OFFICE 



969-7700 



Kay 29 » 1975 



Dear Mr. 



Lut week* I had a chance to ineet with a nuinber of Title I principals during the 
area principals* meetings. Perhaps you attended the neetlngy and did not comr 
""plete the short form concerning your Interest In exploring possibilities for 
building level evaluation (a copy Is enclosed for your Information) • • 

A nundber of Title I principals have Indicated that they had specific evaluation 
projects In mind for next year. What about Clinton School? Have you planned 
programs or activities next year that you might like to evaluate? 

As a second step of the Principals' Committee project on developing models for 
building level evaluation* interviews will be conducted with every Interested 
Title I principal. ETS, a non-profit organization devoted to measurement* evalu«- 
at ion and research in education* has been contracted by the Minneapolis Board of 
Education to work with the Principals' Committee. During the next week or two* 
you will be contacted by my colleague* Bob Relneke. We would be very grateful if 
you could consider* in the meantime* your own building evaluation needs for next 
jmmx in terms of the following: 



1. The kind of evaluation you prefer. 

2. The program(s) you would like to evaluate. 

3« The data available and the data needed for 
your building. 

4* The people in your school or community who 
should be involved in the planning and data 
collection phases of any evaluation to be 
undertaken. 

5. To whom the findings of any evaluation should 
ba reported. 



Mr. lelneke will explore these issues with you during a brief interview at your 
convenience. We hope to come up with several evaluation models which would ad- 
draas the specific evaluation needs of the Title I principals. 



SlAceraly yours, 



aka 

lacloaure 



Ji^jla Bslat Ph.D. 
Director 



OeO OROVB STRBBT • SVANSTON. ZLL.IMOXS eOSOl 
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Exhibit 3 



PrlncjLpal Interview Schedule 

Date 

1. Principal School Area 

A. Background and Building Data 

1. Number of years as principal of this school: Years 

2. Building Parameters (75-76 estimate): 
a« Number of students 

b. Number of teachers 



c. Support staff (professional) 



d. Support staff (non-professional) 

3. What Title I programs in school? 

a* Sponsored and Initiated by central office 
Title 



Date Began 



Continue 75-76? Yes No 

(1) Number of students involved ^ 

(2) Number of teachers involved 

(3) Number of support personnel 



b. Building level or cooperative (description) 
Title 



Date Began 



Description 



(1) Number of students involved 

(2) Number of teachers involved 

(3) Number of support personnel 
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Exhibit 3 - continued 
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Exhibit 3 - continued 

3 

2. Describe current in-house evaluative procedures: 



3. To what extent are internal and external evaluation requirements 
compatible (overlap)? 



4« In view of your concerns/decisions and attendant information 
requirements 9 what kind of evaluation effort would you envision 
which might best serve those needs? 



5. To what extent to you feel that your evaluation requirements 

are similar to those of other K-6 schools? Cooperative programs? 
Which schools? 



6. To what extent are community groups involved in Title I programs? 
To what extent should they be involved? 



C* Evaluation Barriers 

1« What potential barriers do you feel would need to be overcome In 
order to Implement a building level evaluation which adequately 
would serve your decision needs? How might each be handled? 

a« Free response: 

Barrier How Handled 
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Exhibit 3 - continued 

4 



b. Prompt 

Barrier Yes-No Hov Handled 

(1) Teacher resistance ' 

(2) Student resistance 

(3) Lack of trained personnel 

(4) External control 

requirements * 

(5) Insufficient time 

available 

(6) Lack of data collection/ 

analysis capability ' 

(7) External reporting 
requirements 

Who do you feel might properly share in the planning and 
conduct of a building level evaluation? 

List: 



What external services do you think vould be required to 
successfully implement a building level evaluation plan in 
your school? 

a. Free response: 



b. Prompt T«8*No 

(1) Planning consultant . 

(2) Data analysis capability 

(3) Externally developed data 
collection instruments 

(4) Evaluation director 

(5) Evaluation coordinator 

(6) Evaluation resource person 
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No. of days 
required /mo. 
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Exhibit 3 - continued 



Dissemination of Findings 

!• To whom should evaluation findings be disseminated? 
a. Free response 



b« Prompt 

Audience Amount of Information 

Principal 

Teachers 

Students 

Parents 

Super Int endent s 

Board 

State government 

Federal government 

Research 

2. Who should decide to \ahom and to what extent results are 
distributed? 



Assuming adequate resources were available, when would it be 
feasible to begin a building level evaluation at your school? 
and what specific Title I program would you want to assess? 
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